A Hierarchal Planning Framework for AUV Mission Management in a Spatio-Temporal Varying Ocean

The purpose of this paper is to provide a hierarchical dynamic mission planning framework for a single autonomous underwater vehicle (AUV) to accomplish task-assign process in a limited time interval while operating in an uncertain undersea environment, where spatio-temporal variability of the operating field is taken into account. To this end, a high level reactive mission planner and a low level motion planning system are constructed. The high level system is responsible for task priority assignment and guiding the vehicle toward a target of interest considering on-time termination of the mission. The lower layer is in charge of generating optimal trajectories based on sequence of tasks and dynamicity of operating terrain. The mission planner is able to reactively re-arrange the tasks based on mission/terrain updates while the low level planner is capable of coping unexpected changes of the terrain by correcting the old path and re-generating a new trajectory. As a result, the vehicle is able to undertake the maximum number of tasks with certain degree of maneuverability having situational awareness of the operating field. The computational engine of the mentioned framework is based on the biogeography based optimization (BBO) algorithm that is capable of providing efficient solutions. To evaluate the performance of the proposed framework, firstly, a realistic model of undersea environment is provided based on realistic map data, and then several scenarios, treated as real experiments, are designed through the simulation study. Additionally, to show the robustness and reliability of the framework, Monte-Carlo simulation is carried out and statistical analysis is performed. The results of simulations indicate the significant potential of the two-level hierarchical mission planning system in mission success and its applicability for real-time implementation. Keywordsmission planning, optimization, autonomous underwater vehicle, autonomy, dynamic current, spatio-temporal ocean Autonomous operation of AUVs in a vast, unfamiliar and dynamic underwater environment is a complicated process, especially when the AUV is obligated to react to environment changes, where usually a-priory information is not available. Recent advancements in sensor technology and embedded computer systems has opened new possibilities in underwater path planning and made AUVs more capable for handling complicated long range underwater missions. However, there still exist major challenges for this class of the vehicle, where the surrounding environment has a complex spatio-temporal variability and uncertainty. Ocean current variability affect vehicle’s motion, for example it can perturb its safety by pushing that to an undesired direction (Zeng et al., 2014). Consequently, this variability can also have a profound impact on vehicle’s battery usage and its mission duration. The robustness of a vehicle’s path planning to this strong environment variability is a key element to its safety and mission performance. Thus, robustness of the trajectory planning to current variability and terrain uncertainties is essential to mission success and AUV safe deployment. On the other hand, an AUV should carry out complex tasks in a limited time interval. However, existing AUVs have limited battery capacity and restricted endurance, so they should be capable of managing mission time. Obviously, a single AUV is not able to meet all specified tasks in a single mission with limited time and energy, so the vehicle has to effectively manage its resources to perform effective persistent deployment in longer missions without human interaction. In this respect, time management is a fundamental requirement toward mission success that tightly depends on the optimality of the selected tasks between start and destination point in a graph-like operation terrain. Hereupon, design of an efficient mission planning framework considering vehicle’s availabilities and capabilities is essential requirement for maximizing mission productivity. Many efforts have been devoted in recent years for enhancing an AUV’s capability in robust motion planning and efficient task assignment. Although some improvement have been achieved in other autonomous systems; there are still many challenges to achieve a satisfactory level of intelligence and robustness for AUV in this regard. AUVs capabilities in handling mission objectives are directly influenced by routing and task arrangement system performance. Effective routing has a great impact on a vehicle’s time management as well as mission performance by appropriate selection and arrangement of the tasks sequence. Many attempts have been carried out in scope of single or multiple vehicles routing for different purposes. Route planning problems usually is referred as finding shortest paths in a graph-like network such as modelling the transportation network (Geisberger, 2011; Kosicek et al., 2012). The main issue that should be covered by route planning system is to direct vehicle(s) to its destination in a network while providing efficient maneuver and optimizing travel time. Route planning on multi-agent decisions is implemented by (Dominik et al, 2011) for the purpose of transport planning, where the agent decides between distributing orders to customers, traversing edges, competing vendors, increasing production, etc. Zou et al.,(2007) investigated application of Genetic Algorithm(GA) in dynamic route guidance system (Zou et al., 2007). A behavior-based controller coupled with waypoint tracking scheme is introduced by (Karimanzira et al., 2014) for AUV guidance in large-scale underwater environment. An integrated mission assignment and routing strategy is proposed in (Yan et al., 2014) to serve the AUVs routing problem in order to deliver customized sensor packages to mission targets at scattered positions, while minimising total energy cost in the presence of ocean currents. The AUV routing problem is investigated with a Double Traveling Salesman Problem with Multiple Stacks (DTSPMS) for a single-vehicle pickup-and-delivery problem by minimizing the total routing cost (Iori and Ledesma, 2015). A large scale route planning and task assignment joint problem related to the AUV activity has been investigated in (M.Zadeh et al., 2015) by transforming the problem space into a NP-hard graph context and using the heuristic search nature of Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) to find the best waypoints. Later on, the same concept is extended by (M.Zadeh et al., 2016a) to AUV routing in a semi dynamic network, while performance of the BBO and PSO algorithm is tested on single vehicle’s routing approach. The real-time performance of the application is usually overshadowed by growth of the graph complexity or problem search space. Growth of the search space increases the computational burden that is often a problematic issue with deterministic methods such as mixed integer linear programming (MILP) proposed by (Yilmaz et al., 2008) for governing multiple AUVs. In terms of task assignment, many typical AUV missions are limited to executing a list of pre-programmed instructions and completing a predefined sequences of tasks. The majority of the mentioned research particularly focuses on task and target assignment and time scheduling problems without considering requirements for vehicle’s safe deployment or quality of its motion in presence of environmental disturbances. A vehicle’s safe and confident deployment is a critical issue that should be taken into consideration at all stages of the mission in a vast and uncertain environment. In the rest of this section, existing AUV trajectory/path planning approaches are discussed, which are more concentrated on vehicles deployment encountering dynamicity of the operating environment Various strategies have been developed and applied to the AUV path-planning problem in recent years. Willms and Yang (2006) proposed a real-time collision-free robot path planning based on a dynamic-programming shortest path algorithm. A sliding wave front expansion algorithm applying continuous optimization techniques has been presented by (Soulignac 2011) for AUVs path planning in presence of strong current fields. Earlier proposed methods (Willms and Yang 2006; Jan et al., 2008) are capable of providing optimum path planning for AUV using previous information for replanning process, which is computationally reasonable in generating accurate local trajectories; however, they modelled the environment as a 2D space, which is inefficient for application of the AUVs, as a 2D representation of a marine environment doesn’t sufficiently embody all the information of a 3D ocean environment and the vehicle’s six degree of freedom. Jan et al.,(2008) investigated higher geometry maze routing algorithm for optimal path planning and navigating a mobile rectangular robot among obstacles. Nevertheless, this strategy may not be appropriate for AUV dynamic environments where the current field changes continuously during the mission. The evolution-based strategies like Differential Evolution (DE) (M.Zadeh et al., 2016b), GA (Alvarez et al., 2004; Roberge et al., 2013) and PSO (Witt and Dunbabin 2008) are another approach that has been applied successfully to the path planning problem and are fast enough to satisfy time restrictions of the real-time applications. A real-time online evolution based path planner was developed for AUV rendezvous path planning in a cluttered variable environment, in which the performance of four evolutionary algorithms of Firefly Algorithm (FA), BBO, DE, and PSO is tested and compared in different scenarios (M.Zadeh et al., 2016c). A Quantum-based PSO (QPSO) was applied by (Fu et al., 2012) for unmanned aerial vehicle’s path planning, in which only the off-line path planning in a static known environment was implemented, which is not enough to cover dynamicity of the underwater environment. Later on, this algorithm was employed by (Zeng et al., 2014) for AUV’s on-line path planning in dynamic marine environment. Although various path planning techniques have been suggested for autonomous vehicles, AUV-oriented applications still have several difficulties when operating across a large-scale geographical area. The recent investigations on path planning that incorporate variability of the environment have assumed that planning is carried out with perfect knowledge of probable future changes of the environment (Garau et al.,2009; Smith et al., 2010), while in the reality, accurate prediction of the environmental events such as currents or obstacles state variations is impractical specially in longer operations. Even though available ocean predictive approaches operate reasonably well in small scales and over short time periods, they produce insufficient accuracy to current prediction over long time periods in larger scales, specifically in cases with lower information resolution (Hollinger et al., 2013). Moreover, current variations over time can affect moving obstacles (or waypoints in some cases) and drift them across a vehicle’s trajectory; therefore, the planned trajectory may change and become invalid or inefficient. Proper estimation of the events in such a dynamic uncertain terrain in long range operations, outside the vehicle’s sensor coverage, is impractical and unreliable. This becomes even more challenging in larger dimensions, when a large data load of variation of whole terrain condition should be computed repeatedly any time that path replanting is required, which is computationally inefficient and unnecessary as only awareness of environment changes in vicinity of the vehicle is enough. As mentioned earlier, the path planning problem principally deals with the quality of a vehicle’s motion between two points and it is not an appropriate strategy to carry out vehicle’s task assignment and mission timing in a graph-like terrain. On the other hand, vehicles routing strategies are not flexible like path planning methods in terms of handling environment sudden changes, but they give a general overview of the area that an AUV should fly through (general route), which means reducing the operation area to smaller operating zone for vehicle’s deployment. To satisfy the addressed challenges and to produce a reliable mission plan for a large scale time-varying underwater environments, this paper proposes a reactive hybrid framework that comprises an efficient mission planning system combined with real-time path planning that improves a vehicle’s ability to complete as much of its mission as possible within the time available. Furthermore, a path planner is designed in a smaller scale to concurrently plan trajectory between waypoints included in the task sequence. The path planner operates in the context of the mission planner in a manner to be fast enough to handle unforeseen changes and regenerates an alternative trajectory that safely guides the vehicle through the specified waypoints with minimum time/energy cost. A constant interaction exists between high-low level planners across small and large scale. This paper is a continuation of previous research (M.Zadeh et al. 2016g) in which the environment modeled to be more realistic comprising uncertainty of moving/afloat objects and dynamic multiple-layered time varying ocean current; accordingly, the path planner in current study is facilitated with dynamic re-planning capability, which have not been addressed in the previous study. The reactive hybrid model decides whether to carry out the path re-planning or mission replanning procedure according to the raised situation. The path re-planning is performed to cope with dynamic changes of the operating environment over time. On the other hand, mission re-planning is performed to manage the lost time in cases that the path planner process takes longer than expectation. The proposed re-planning procedure in both of the mission and path planners improve the robustness and reactive ability of the AUV to the environmental changes and enhance its performance in accurate mission timing. Both of the planners operate individually and concurrently while sharing their information. Parallel execution of the planners speed up the computation process. In the core of the proposed strategy, both mission planner and local path planners make use of the Biogeography-based Optimization (BBO) algorithm. The argument for application of BBO in solving Non-deterministic Polynomial-time (NP) hard problems is strong enough due to its remarkable competency in scaling with multi-objective and complex problems. In this algorithm solutions of one generation are transferred to the next and never discarded but modified. This characteristic of the BBO enhances its exploitation ability. Solving NP-hard problems is computationally challenging and currently there is no polynomial time algorithm to handle a NP-hard problem of even moderate size. Furthermore, finding a pure optimum solution is only possible when the environment is fully known and no uncertainty exists. The modeled underwater environment in this paper corresponds to a highly dynamic uncertain environment. The BBO is one of the fastest meta– heuristics algorithms introduced for solving NP-hard complex problems. Although the captured solutions do not necessarily correspond to a pure optimal solution, controlling the computational time is more preferable in this research due to the realtime application of the AUV operations; hence the BBO is employed to find feasible and near optimal solutions in competitive CPU time. More importantly, the main contribution of this paper is the proposed reactive hybrid structure which reduces run time by splitting the operation terrain to smaller zones for the path planner and also its comprehensive application for both mission management and path planning regardless of the employed algorithm, which has not been addressed before. The organization of the paper is as follows. The ocean terrain is mathematically modeled and presented by Section 2. In Section 3, application of the BBO Algorithm on Mission Planning and Local Path Planning problem is discussed. The proposed autonomous/reactive system is implemented in MATLAB®2016 and its performance is statistically analyzed and a discussion of the simulation results is provided by Section 4. In Section 5 conclusions are presented. 2 Mathematical Representation of the Underwater Terrain Existence of prior information about the terrain, location of coasts and static obstacles as the forbidden zones for deployment, position of the start, target, and waypoints in operating area promotes AUV’s capability in robust path planning. To model a realistic marine environment, a three dimensional terrain in scale of {10×10 km (x-y), 1000 m(z)} is considered based on realistic example map presented by Fig.1, in which the operating field is covered by uncertain static-moving objects, several fixed waypoints and variable ocean current. A k-means clustering method is employed to classify the coast, water and uncertain area of the map. To this purpose, a large map with the size of 1000×1000 pixels is conducted that presents 10 km square area for the mission planner and a smaller part of the map in size of 350×350 pixels that corresponds to 3.5 km square is selected to implement and test the local path planner’s performance. In this map each pixel corresponds to 10 m square space. On the other hand, the operation environment is modeled by geometrical network. Therefore, nodes in the network present waypoints in operation area in which various tasks assigned to passible distance between connected nodes in advance. Hence, every edge has weight and cost that is combination of tasks priority, tasks completion time, length of the connection edges and the time required for traversing the edges. The waypoints location are randomized once in advance according to ~U(0,10000) for Px,y and ~U(0,1000) for Pz in the joint water covered sections of the map. The clustered sections in both small and large scale maps are presented in Fig.1. The clustered map is converted to a matrix format, in which the matrix size is same to map’s pixel density. The corresponding matrix is filled with value of zero for coastal sections (forbidden area in black color), value of (0,0.35] for uncertain sections (risky area in grey color), and value of one for water covered area as valid zone for vehicles motion that presented by white color on the clustered map. The utilized clustering method is capable of clustering any alternative map very efficiently in a way that the blue sections sensed as water covered zones and other sections depending on their color range categorized as forbidden and uncertain zones. Fig.1. The original and clustered map for both mission planner and the local path planner. In the clustered map In addition to offline map, three types of static uncertain static obstacles, afloat and self-motivated moving objects are considered in this research to cover different possibilities of the real world situations. Movement of afloat objects are considered to be affected by current force, where the self-motivated moving obstacles are considered to have a motivated velocity in additional to current effects that shift them from arbitrary position A to a random direction. Obstacle’s velocity vectors and coordinates can be measured by the sonar sensors with a specific uncertainty that modeled with a Gaussian distribution; hence, each obstacle is presentable by its position (Θp), dimension (diameter Θr) and uncertainty ratio. Position of the obstacles initialized using normal distribution of N�~(0,σ) bounded to position of two targeted waypoints by the path planner (e.g. Px,y,z<Θp<Px,y,z), where σ≈Θr. Further detail on modeling of different obstacles can be find at (M.Zadeh et al. 2016d; 2016e). Beside the uncertainty of operating field, which has been taken into account in the modeling the operation terrain, water current can also have a detrimental effect on mission objectives. These issues, therefore, need to be address thoroughly in accordance with the type and range of the mission. The ocean current information can be captured from remote observations or can be obtained from numerical estimation models. Usually motion of the AUV is considered on the horizontal plane, because vertical motions in ocean structure are generally negligible due to large horizontal scales comparing to vertical (Garau et al., 2006). For the purpose of this research, a 3D turbulent time-varying current has been applied that is generated using a multiple layered 2D current maps, in which the current circulation patterns gradually change with depth. The current map gets updated by applying each of the corresponding layers in a specific time step. The current map in this research modeled by:                              , , , , : ) , ( ~ ; 0 0 ) π 2 det( γ ) ( ) π( 2 ) ( ) ( ) π( 2 ) ( ) (